Skip to content

feat(pokestop): consume Golbat /api/pokestop/available with endpoint-authoritative fallback (draft)#1227

Draft
jfberry wants to merge 7 commits into
WatWowMap:developfrom
jfberry:feat/pokestop-available-consumer
Draft

feat(pokestop): consume Golbat /api/pokestop/available with endpoint-authoritative fallback (draft)#1227
jfberry wants to merge 7 commits into
WatWowMap:developfrom
jfberry:feat/pokestop-available-consumer

Conversation

@jfberry

@jfberry jfberry commented Jul 14, 2026

Copy link
Copy Markdown

Summary

Adds a ReactMap consumer for Golbat's new GET /api/pokestop/available (Golbat draft PR UnownHash/Golbat#383). A scanner source that has a Golbat endpoint now fetches the pokestop filter/available list from Golbat instead of running ~30 DISTINCT/GROUP BY SQL queries every 15 minutes — mirroring how Pokemon.getAvailable already consumes /api/pokemon/available. Everything else about pokestops (map markers, popups, search, submissions) is unchanged.

A source can now be dual (endpoint + DB creds): the migrated query (getAvailable) uses the endpoint, while un-migrated queries fall back to this.query() on the bound DB — so a Golbat pokestop endpoint is safe to enable without touching those features.

How it works

  • server/src/models/pokestopAvailableMapper.js (new) — a pure mapper reproducing the SQL path's filter-key formulas exactly (quest rewards q/d/p/m/c/x/u + pokémon, invasions i/b, confirmed rocket a, lures l, showcases f/h, plus per-reward conditions). Its process() helper is byte-identical to the SQL path's.
  • Pokestop.getAvailable wiring — an if (mem) endpoint branch that fetches, validates the response shape, and maps it. On failure (503 / fort_in_memory off, network error) it falls through to the SQL block: a dual source runs the SQL fallback on its bound knex; a pure-endpoint source has no knex, so it's dropped by the caller's Promise.allSettled.
  • Dual endpoint+DB sources (DbManager) — a schema carrying both endpoint and DB creds registers the endpoint and builds a knex connection; the endpoint context (mem/secret/httpAuth) is overlaid onto the schema-checked DB context. Migrated methods read mem; un-migrated ones use the bound DB. Pure-endpoint and pure-DB schemas behave exactly as before.
  • applyRocketPokemonFallback (shared helper) — the config-derived rocket-pokémon a-keys (from state.event.invasions[*].encounters, gated by fallbackRocketPokemonFiltering, default on) are emitted from one helper used by both the SQL and endpoint paths, so those keys are identical by construction.
  • TypesAvailablePokestops (+ members); ApiEndpoint.httpAuth typed; DbConnection gains optional endpoint/secret/httpAuth for the dual shape.

Configuring & testing

Requires the Golbat side up: [general] fort_in_memory = true and the API secret matching ReactMap's. Sanity-check Golbat directly:

curl -H "X-Golbat-Secret: <secret>" http://<golbat-host>:<port>/api/pokestop/available   # 200 + {quests,invasions,lures,showcases}

Most instances already run Golbat for pokémon: a scanner DB source serving ["gym","pokestop","spawnpoint",...] plus a separate pure-endpoint golbat source for ["pokemon","device"]. To route the pokestop available list through Golbat while keeping pokestop markers/popups/search on the DB, just add the two endpoint fields to your existing scanner DB source (the one whose useFor includes pokestop):

// config/local.json → database.schemas[]  (your existing scanner DB source)
{
  "host": "127.0.0.1", "port": 3306,
  "username": "...", "password": "...", "database": "golbat_db",
  "endpoint": "http://{golbat_address}:{golbat_port}",   // ← add (same as your pokemon golbat source)
  "secret": "<golbat api secret>",                       // ← add
  "useFor": ["gym", "pokestop", "spawnpoint", "weather", "station", "..."]
}

That's the whole change — the source becomes dual. Only pokestop consumes the endpoint (getAvailable); gym/station/etc. don't check mem, so they keep using the DB; and pokestop markers/popups/search/submissions fall back to this.query() on the same DB. Your pure-endpoint pokémon source is untouched. (If you'd rather scope it tighter, split a useFor: ["pokestop"]-only copy of the DB source with the endpoint added and remove pokestop from the original — but it isn't necessary.)

Behavior:

  • pokestop filter options ← Golbat /api/pokestop/available (offloads the ~30-query, 15-min SQL)
  • pokestop markers / popups / search / submissions ← the bound DB (unchanged)
  • Golbat 503 → getAvailable falls back to SQL on the DB

Verify: filter options populate and markers/popups/search still work; cross-check the filter list against your pre-change list (the form_id 0 / type-20 golden check, on live data).

Verification

Per maintainer preference, no test suite was added (this repo has none). Correctness was established by a line-by-line golden comparison of the mapper against the SQL key-building (all reward-type formulas, the -form rule, the u-set, i/b, showcases, lures, with_ar merge confirmed to match), throwaway sanity scripts, an end-to-end output-shape check (mapper output is structurally identical to the SQL return and consumed identically by DbManager.getAvailable + filters/builder/pokestop.js), and eslint/prettier clean (adding the dual-source types reduced the pre-existing tsc error count by one).

Residual golden check (before enabling in prod)

Diff mapAvailablePokestops(liveGolbatResponse) vs the SQL getAvailable on the same data: form_id === 0 → bare <id>; type-20 real data (m<id> vs u20); a-key slot 2/3 when fallbackRocketPokemonFiltering is off. If any diverge, the cleaner fix is Golbat-side (convey null form / a type-20 sentinel) — coordinate via #383.

Follow-ups (deferred, non-blocking)

  • Extract a shared evalQuery util — Pokestop.evalQuery is a ~47-line copy of Pokemon.evalQuery (task-scoping artifact).
  • Optionally migrate getOne (simple, endpoint exists) / getAll (complex, needs incidents added to Golbat's scan) to shave DB load — not needed for correctness thanks to the dual-source DB fallback.
  • fetchJson logs the request payload (incl. base64 auth header) on every 503 — pre-existing, worth quieting.

Depends on

Golbat UnownHash/Golbat#383 (the /api/pokestop/available endpoint). Draft until that lands and the golden check is run.

🤖 Generated with Claude Code

jfberry and others added 6 commits July 14, 2026 17:32
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pure, dependency-free mapper that reproduces Pokestop.js's SQL-derived
{ available, conditions } filter-key shape from the structured tuples
returned by Golbat's GET /api/pokestop/available, so a future endpoint
consumer can replace the SQL block key-for-key.
…tardust

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wires Pokestop.getAvailable() to Golbat's GET /api/pokestop/available
when a source has `mem` set (an endpoint source), using the Task 2
mapper to build the same {available, conditions} shape the SQL path
produces. Falls back to the existing SQL block unchanged on a
non-2xx response, a network/timeout error, or any thrown error;
MAD/DB sources (mem: '') skip the branch entirely and always run SQL.

Extracts the config-gated `fallbackRocketPokemonFiltering` a${id}-${form}
backfill (previously inline in the SQL rocketPokemon case) into a shared
applyRocketPokemonFallback() helper so both the endpoint and SQL paths
stay byte-identical. Adds a Pokestop.evalQuery static mirroring
Pokemon.evalQuery, since Pokestop extends Model directly and had no
endpoint-fetch helper of its own.
… SQL fallback path)

Endpoint sources (mem truthy) have no bound knex, so on
/api/pokestop/available failure the previous fallthrough to the SQL
block threw on this.query(), was swallowed by Promise.allSettled, and
silently contributed zero filter keys. Endpoint sources are now
endpoint-authoritative: on failure they return a clean empty
{ available: [], conditions: {} } instead of falling through, matching
Pokemon.getAvailable. The SQL block is reached only for mem:'' (DB/MAD)
sources.
@jfberry

jfberry commented Jul 14, 2026

Copy link
Copy Markdown
Author

Status of this branch: does not operate in its current form as only the get available branch is ready; examining whether we should implement other api calls for pokestops here, or whether to allow a hybrid option

A source with both a Golbat endpoint and DB creds now registers the
endpoint AND builds a knex connection: migrated queries (Pokestop
getAvailable) use the endpoint, un-migrated ones (getAll/getOne/search/
submissions) fall back to this.query() on the bound DB — so a Golbat
pokestop endpoint no longer breaks map markers/popups/search.

- DbManager: keep the knex connection for endpoint schemas that also
  carry DB creds; overlay mem/secret/httpAuth onto the schemaChecked
  context so isMad/has* flags survive.
- Pokestop.getAvailable: on endpoint failure fall through to the SQL block
  (a dual source runs it on the bound knex; a pure-endpoint source throws
  and is dropped by allSettled) instead of returning empty.
- types: ApiEndpoint.httpAuth typed; DbConnection gains optional
  endpoint/secret/httpAuth for the dual shape.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant